Historical Annotations and Humble Databases

As databases and systems of interrelated databases become more common and more complex, our society comes to rely crucially on their accuracy and intelligibility. Stories abound of false information irrevocably ruining someone’s credit ratings, employment records, or worse. In trying to deal with such tragedies, society finds that computer systems are designed with the view that they are monolithic, infallible sources of information. This leads to great disrespect and growing resentment of these large information systems. If we are to justify our reliance on these systems while avoiding society’s censure, we can take either one of two paths. We might make databases responsible for their contents, but this is impossible with current technology. More practically, with current techniques we can make databases explicitly defer all responsibility to humans. What we can do is to have database systems keep historical information about their inputs and about the computations they perform. This historical information can then be used to construct explanations or justifications of each database entry so that errors can be traced to bad inputs, to faulty programs, or to other databases in a distributed system. In this way, the computer can be prepared with the fact of its own fallibility and irresponsibility, and can help track down its own problems and those of its users. While this may not render intelligible the enormous systems of programs involved, at least their effects will have been isolated to some extent. This may add considerable cost in implementing database systems, but we suggest that in society’s view it is too expensive not to provide these facilities. With this motivation, we briefly outline methods for constructing such “humble” databases. The basic idea is simple. The main objective is that no information begin with the database; all information can be traced to some identifiable external source. To achieve this objective, two steps are required. First, each new piece of information entered into the system is annotated with its source. For example, the source specification might mention the time and location of entry, the person entering the data, the physical form number and the location of the form in physical files, if any, and perhaps other relevant information. In highly influential databases in which the security of the system is very important, unforgeable identification codes for the person involved might be used. Second, whenever computations are performed on the database, again each record in the resulting database is annotated with its source, namely the records used in its computation, the time of the computation, the version of the program and compiler used, the computer used, and other relevant information. For example, when two files are merged, each record in the resulting file points back to some record in one of the merged files. When records in one file are used to update those in another (or in itself), the updated records similarly point to the origin records, time, program, etc. (In fact, these annotations might well exist for each field of the records, to show just what other fields of the records were involved in computing the annotated field. This would help to unravel the actions of complex programs as well.)